1. Field of the Invention
The present invention relates to video signal processing apparatuses, video signal processing methods, video signal processing programs, and storage media storing the video signal, processing programs, and is applicable to processing video signals generated by 2-3 pulldown. When temporal features of an input video signal are filtered to suppress variation in the temporal features, different filtering characteristics are used depending on the attribute of the input video signal. With this arrangement, even in processing of an input video signal generated by 2-3 pulldown, for example, stable video signal processing can be realized by effectively preventing deterioration of image quality due to a temporary change in the temporal features associated with a scene change.
2. Description of the Belated Art
In various video signal processing such as image enhancement, noise reduction, IP (interlace-progressive) conversion, and moving image compression, processing to be performed on an input signal is dynamically switched on the basis of temporal features of an input video signal. Temporal features herein refer to features of a video signal which change across consecutive fields or frames and are represented by a value indicating, for example, the amount of noise and the amount of motion. Specifically, the temporal features are calculated in units of pixels, blocks set in a field or a frame, fields or frames, or a predetermined number of fields or frames, etc. For example, the temporal features may be an inter-field or inter-frame difference in pixel values, a sum of the absolute values of the inter-field or inter-frame difference, a variance and a mean value of the difference, a pixel-based motion vector, a block-based motion vector, an encoding difficulty, etc.
FIG. 12 is a block diagram illustrating a basic configuration of a video signal processing apparatus which utilizes temporal features. In FIG. 12, a video signal processing apparatus 1 includes a temporal feature detecting unit 2, a control information generating unit 3, and a video signal processing unit 4. The temporal feature detecting unit 2 receives an input video signal S1 and detects a temporal feature 32. The control information generating unit 3 processes the temporal feature S2 and outputs control information S3 for controlling operations of the video signal processing unit 4. The video signal processing unit 4 processes the input video signal S1 and outputs an output video signal S4.
When the video signal processing to be performed by the video signal processing unit 4 is processing for reducing incise in the input video signal S1, the control information generating unit 3 processes the temporal feature S2 and generates the information control signal S3 on the basis of the temporal feature S2, so that the effect of the noise reduction processing increases with increasing noise in the input video signal S1. When the video signal processing to be performed by the video signal processing unit 4 is IP conversion, the control information generating unit 3 processes the temporal feature S2 and generates the information control signal S3 on the basis of the temporal feature S2, so that the effect of intra-field or intra-frame interpolation increases with increasing amount of motion contained in the input video signal S1. IP conversion refers to format conversion processing in which a video signal of an interlace format (interlace signal) is converted into a video signal of a progressive format (progressive signal) or vice versa, in this case, the video signal processing unit 4 performs format conversion on a video signal by interpolating a result of inter-field or inter-frame interpolation processing and a result of intra-field or intra-frame interpolation processing. The control information generating unit 3 controls an interpolation ratio applied to the interpolation operation using the information control signal S3.
In the video signal processing apparatus 1 having the above configuration, the temporal feature S2 detected by the temporal feature detecting unit 2 varies in time as illustrated in FIG. 13. Thus, the effect of processing of the output, video signal S4 varies in time along with the transition of the temporal feature S2. As a result, the visual quality of an image corresponding to the output video signal S4 is deteriorated.
To address this, a video signal processing apparatus 11 may be provided as illustrated in FIG. 15. Unlike the video signal processing apparatus 1 shown in FIG. 12, the video signal processing apparatus 11 includes a temporal feature smoothing unit 12. The temporal feature smoothing unit 12 smoothes a temporal feature S2 detected by a temporal feature detecting unit 2 and generates a smoothed temporal feature S5, as illustrated in FIG. 13. Using the smoothed temporal feature S5, a control information generating unit 3 generates control information S3. As a result, as illustrated in a broken line in FIG. 14, the video signal processing apparatus 11 prevents an abrupt temporary change in the effect of various processing in an output video signal S4. Note that the temporal feature smoothing unit 12 may be implemented as, for example, an IIR (infinite impulse response) filter or an FIR (finite impulse response) filter.
Techniques related to such video signal processing based on a temporal feature have been developed. For example, Japanese Unexamined Patent Application Publication No. 2002-15327 discloses a method in which a level of image enhancement is changed on the basis of a result of determination using a temporal feature as to whether or not an image is a natural image. Japanese Unexamined Patent Application Publication No. 2004-323635 discloses a technique relating to noise reduction processing. In this technique, a motion vector is used as a temporal feature and a reference value in measurement of a noise level is generated. In addition, Japanese Unexamined Patent Application Publication No. 2003-209716 discloses a method in which a noise level is measured from a temporal feature. Further, regarding IP conversion, Japanese Unexamined Patent Application Publication No. 2007-82040 discloses a method in which a field sequence associated with a 2-3 pulldown process is detected from a temporal feature and processing is switched in accordance with a detection result.
However, when the input video signal S1 is processed by smoothing the temporal feature S2 using the temporal feature smoothing unit 12, the smoothed temporal feature S5 may fluctuate over a plurality of fields or frames due to a temporary change in the temporal feature S2 which is associated with a scene change or the like, as illustrated in FIG. 16 and FIG. 17. This temporary change in the temporal feature S2 affects processing of a plurality of fields or frames, resulting in deterioration of image quality. Mote that FIG. 16 and FIG. 17 illustrate cases where an IIR filter and an FIR filter are implemented, respectively, as the temporal feature smoothing unit 12.
In addition, as illustrated in FIG. 18, the input video signal may be an interlace video signal generated by 2-3 pulldown. In this case, in the input video signal S1, consecutive even fields or odd fields include fields corresponding to the same original frame, which are illustrated as fields A1 and fields C2 in FIG. 18. In processing of such an input video signal S1, when the temporal feature S2 across different frames are detected, no change in the temporal feature S2 will be detected between the fields A1 and between the fields C2. This results in an abrupt temporary decrease in the temporal feature S2 as illustrated in FIG. 19 and FIG. 20 which, for comparison, correspond to FIG. 16 and FIG. 17, respectively. Thus, this also affects processing of a plurality of fields and frames, as in the case of a scene change, resulting in deterioration of the image quality.